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1.0  INTRODUCTION 


This  is  the  first  quarterly  status  report  on  a program  for  Recognition 
Technology  for  A Smart  Sensor , being  conducted  by  the  Westinghouse  Systems 
Development  Division  for  the  Computer  Science  Center,  University  of  Maryland. 

The  program  contains  three  phases,  as  follows: 

Phase  I Task  and  Technology  Review  (3  months) 

Phase  II  Algorithm  Selection  and  Test  (9  months) 

Phase  III  Hardware  Development  (9  months) 

This  report  covers  the  Phase  I effort.  In  addition,  it  describes 
initial  activity  on  algorithms  implementation  which  is  a part  of  Phase  II. 

The  report  was  prepared  by  Mr.  Tom  Willett  of  Westinghouse.  The  Westinghouse 
program  manger  is  Dr.  Glenn  E.  Tisdale. 

During  the  quarter,  eight  meetings  were  held  between  members  of  the  Mary- 
land and  Westinghouse  teams.  Mr.  John  Dehne,  NVL  Program  Manager,  maintained 
close  contact  with  the  program,  and  attended  some  of  the  meetings.  The  meetings 
included  a description  by  Maryland  of  the  nature  of  the  algorithms  which  are 
under  consideration  for  the  cueing  function,  together  with  preliminary  test 
results,  presentations  by  Westinghouse  on  the  nature  of  automatic  cueing, 
and  the  limitations  involved  in  the  hardware  implementation  of  cueing  algorithms. 
This  material  is  summarized  in  this  report. 


During  the  quarter,  Westinghouse  aupplled  to  Maryland  a taped  data  base 
of  digitized  forward-looking  infra-red  images,  together  with  ground  truth,  for 
use  in  testing  algorithms.  The  taped  data  base  had  been  furnished  Westinghouse 
by  the  Night  Vision  Laboratory  of  the  U.  S.  Arny  Mobility  Equipment  Research  and 
Development  Command  under  another  contract. 

1.1  DEFINITION  OF  THE  AUTOMATIC  TARGET  CUEING  FUNCTION 

Before  discussing  the  design  goals  and  hardware  implementation  for  an 
automatic  target  cueing  system,  it  is  useful  to  examine  its  character  and  purpose. 

As  the  tools  used  in  reconnaissance  and  target  acquisition  operations  con- 
tinue to  improve,  it  is  becoming  increasingly  clear  that  man  himself  is  a per- 
formance bottleneck.  Sensors  provide  greater  resolution,  dynamic  range,  and 
speed;  and  the  techniques  that  make  effective  use  of  sensor  information,  such 
as  communication  links,  and  weapon  delivery  systems,  are  also  being  advanced. 
However,  acquisition  of  target  material  largely  depends  on  the  ability  of  a human 
operator  to  scan  sensor  imagery  in  real  time.  If  he  misses  targets,  or  identifies 
them  incorrectly,  mission  performance  is  degraded. 

One  way  to  improve  this  situation  is  to  provide  the  human  with  cues  to 
locate,  and  possibly  identify,  targets  on  his  displays.  Such  cues  might  originate 
with  a priori  information,  or  from  the  detection  of  peculiar  target  conditions 
such  as  motion,  hot  spots,  or  electromagnetic  radiation.  However,  a more  general 
approach  is  to  apply  automatic  recognition  techniques  to  the  imagery.  As  shown 
in  Figure  1-1,  the  image  processor  serves  as  an  information  filter  on  the  image, 
alerting  the  human  to  the  presence  of  potential  targets,  possibly  by  audible 
signals  initially,  and  then  by  providing  visual  cues  or  overlays  on  his  display. 

In  this  process,  the  final  verification  of  target  identity  is  reserved  for  the 
human  operator.  The  image  processor  serves  to  assist  him. 
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Figure  1-1.  Information  Extraction  and  Automatic  Cueing 
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Automatic  cueing  can  be  carried  out  either  in  airborne  or  ground  locations. 

In  the  airborne  situation,  the  operator  views  a CRT-type  display  for  acquisition 
of  targets  on  a real-time  basis.  His  determination  may  result  in  action  in  a 
matter  of  seconds,  either  offensive  or  defensive.  On  the  ground,  interpretation 
may  be  required  in  real-time,  or  on  a more  relaxed  base's . In  the  proposed 
operation  of  remotely  piloted  vehicles  (RPV's),  for  example,  a video  link  may 
be  used  to  obtain  a CRT  presentation  at  a ground  station  of  the  output  of  a 
sensor  located  on  the  vehicle.  The  problems  for  the  operator  are  somewhat  similar 
to  the  airborne  situation;  however,  his  appraisal  of  the  sensor  image  is  entirely 
limited  to  the  CRT  output.  He  can't  look  at  the  target  area  directly.  On  the 
other  hand,  he  is  not  distracted  by  problems  such  as  vehicle  operation  and 
personal  security.  In  a more  relaxed  mode,  cueing  of  wide  area  high  resolution 
reconnaissance  imagery  might  be  used  to  greatly  reduce  the  time  required  in  scanning 
imagery.  In  this  case,  the  results  of  the  processing  are  required  in  a matter  of 
minutes,  so  as  to  respond  to  the  detection  of  mobile  targets. 

1.2  CONTENTS  OF  THIS  REPORT 

The  report  will  proceed  in  Section  2.0  with  a discussion  of  design  goals  for 
a cueing  system.  The  goals  will  depend  to  an  extent  on  the  nature  of  the  missions 
involved;  accordingly,  some  typical  missions  will  be  listed.  Next,  performance 
goals  will  be  considered,  including  the  desired  detection  rate  for  targets,  the 
acceptable  false  alarm  rate,  and  the  required  speed  of  operation.  The  physical 
characteristics  of  the  system  will  then  be  examined,  followed  by  a discussion  of 
allowable  cost. 

Section  3.0  will  provide  a general  discussion  of  the  constraints  imposed 
on  the  cuer  algorithms  by  requirements  for  high-speed,  low  cost  hardware. 

j 
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Section  4.0  will  introduce  the  approach  to  the  implementation  of  specific 


algorithms  in  circuit  form.  This  work  is  a part  of  Phase  II  of  the  program, 
and  will  continue. 
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2.0  SYSTEM  DESIGN  GOALS 


The  key  considerations  in  the  design  of  an  automatic  target  cueing  system 
are  its  performance,  physical  characteristics,  and  cost.  We  will  deal  with  each 
of  these  areas  in  this  section.  However,  a quantitative  determination  of  design 
parameters  will  depend  on  the  manner  in  which  the  mission  is  implemented.  Such 
implementation  will  he  discussed  first. 

2 . 1 MISSION  IMPLEMENTATION 

As  explained  in  Par.  1.1,  the  target  cueing  function  might  he  performed 
aboard  a vehicle,  or  at  a ground  station  if  imagery  is  relayed  for  analysis.  In 
either  case,  the  performance  goals  will  tend  to  he  comparable.  As  regards 
physical  characteristics  and  cost,  however,  the  vehicle  location  will  provide 
much  tighter  restrictions.  Our  discussion  will  proceed  on  the  basis  of  the 
vehicular  application.  Both  helicopters  and  high-speed  aircraft  are  airborne 
candidates.  The  RPV  image,  on  the  other  hand,  will  be  analyzed  at  a ground 
station;  therefore  the  physical  limitations  within  the  RPV  are  not  a problem. 

As  the  state-of-the-art  in  automatic  target  recognition  develops,  and  high  levels 
of  performance  are  attained,  it  is  anticipated  that  the  human  observer  will 
eventually  be  eliminated  in  some  applications.  For  example,  recognition  equipment 
might  be  placed  aboard  a missile  for  unaided  terminal  guidance.  The  requirement 
for  high  performance,  small  size  and  weight,  low  power  consumption,  and  low 
cost  will  all  apply  in  this  case. 
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2.2  PERFORMANCE  GOALS 

The  key  performance  parameters  are  the  detection  and  recognition  rates  for 
targets  of  interest,  the  false  alarm  rate,  and  the  speed  of  operation  of  the 
cueing  system.  A discussion  of  these  parameters  follows. 

2.2.1  Detection  and  Recognition  Rates 

Detection  of  a target  occurs  when  a target  indication  is  signalled  by  the 
cueing  system  in  response  to  the  presence  of  a target  of  interest.  Correct 
recognition  occurs  when  the  appropriate  target  class  is  selected  from  among 
several  possible  classes.  Detection  and  recognition  performance  is  expressed 
as  a percentage  of  the  targets  actually  present.  Therefore,  in  testing  the 
performance  of  a cueing  system,  it  is  important  that  "ground  truth"  be  available 
for  the  test  data  base. 

What  levels  of  detection  and  recognition  are  required  in  order  that  the 
cueing  system  will  be  cost  effective?  Clearly,  its  use  should  provide  a signi- 
ficant increase  in  the  capability  of  the  human  interpreter.  First,  consider  the 
interpreter’s  situation.  His  task  is  to  locate  and  identify  objects,  such  as 
vehicles  or  boats,  that  generally  occupy  a small  area  in  the  sensor  field  of 
view,  and  that  may  be  imbedded  in  background  clutter  and  noise . If  there  were 
no  clutter  or  noise,  the  need  for  cueing  would  largely  vanish.  There  is  an 
advantage  to  target  acquisition  at  maximum  range,  which  implies  that  recognition 
will  occur  with  a minimum  of  target  resolution.  In  fact,  attempts  must  be  made 
to  infer  the  nature  of  "blobs"  on  the  basis  of  contextual  information.  The 
human  interpreter  of  reconnaissance  imagery  is  limited  in  several  ways: 
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• His  ability  to  serach  imagery  is  limited  by  foveal  vision 
and  reaction  time. 

• He  is  subject  to  eye  fatigue. 

• His  performance  is  degraded  by  the  demands  of  other  tasks 
and  by  his  concern  for  platform  navigation  and  safety. 

What  performance  do  humans  actually  provide?  Although  data  are  limited, 

1 2 

it  would  appear  that  it  is  fairly  low.  Evidence  is  accumulating  ’ that  even 
under  close  scrutiny  and  laboratory  conditions  less  than  half  of  available 
targets  are  detected  or  recognized  by  trained  interpreters.  In  the  aircraft 
environment,  the  situation  is  evidently  worse,  more  like  33$  detection  and 

o 

25$  recognition. 

Consider,  now,  automatic  recognition  by  image  processing.  Image  processing 
and  recognition  machines  offer  several  advantages : 

• They  can  be  wide  band  systems , capable  of  examining  image  elements 
at  megasample  per  second  (TV)  rates; 

• They  may  be  designed  to  operate  on  sensor  video  signals,  thereby 
avoiding  loss  of  resolution  and  dynamic  range  caused  by  displays; 


1.  Jeffrey.  T.E.,  Image  Interpretation  Research,  from  "Pattern  Identification  by 
Man  and  Machine,"  U.S.  Army  Tech.  Memo  17-^8,  Human  Engineering  Laboratories, 
Aberdeen  Proving  Ground,  Md. , 12  December  1968,  P.  35. 

2.  Evans,  S.H.  and  Dansereau,  D.F.,  Relating  Man  to  Automatic  Image  Interpretation: 

It  Pays  to  Know  Your  Competitor,  paper  presented  at  the  EIA  Symposium  on  Auto- 
matic Photointerpretation  and  Recognition,  Washington,  D.C.,  December  7,  1971. 

3.  Airborne  Stano  Systems  Part  II  Test  Report  (U),  U.S.  Army  MASSTER,  Ft.  Hood,  Texas, 
7^544,  July,  1971. 
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• They  are  expendable,  capable  of  being  stockpiled,  and  don't  require  training. 

• They  are  not  subject  to  fatigue  or  distractions. 

What  kind  of  performance  may  be  expected  from  the  machines?  Based  on 
preliminary  test  results,  it  appears  reasonable  to  expect  greater  than  Q0% 
detection  and  60%  recognition  of  targets  in  clutter  under  comparable  conditions. 

These  figures  are  more  than  double  the  figures  for  human  performance  stated  above. 

This  would  indeed  provide  a good  margin  for  cost-effective  performance. 

2.2.2  False  Alarm  Rate 

A false  alarm  occurs  when  an  indication  of  a target  is  signalled  by  the 
cueing  system  when,  in  fact,  no  target  is  present.  The  false  alarm  rate  is 
expressed  on  the  basis  of  a unit  of  elapsed  time  or  area  of  coverage. 

Where  several  different  target  classes  are  involved,  it  is  possible  to  think 
about  a quantity,  P,  , that  represents  detection  probability  averaged  over  all 

USb 

the  target  classes  of  interest;  is  the  false  alarm  counterpart.  P^.a  is  based 
on  one  frame  covering  the  whole  field  of  view. 


The  false  alarm  rate'’  which  the  human  operator  must  combat  is  peculiar  to 
video  displays.  Since  most  false  alarms  are  due  to  ground  clutter,  not  noise, 
they  will  reappear  in  every  frame  until  the  scene  changes.  Once  the  operator 
identifies  a cue  as  a target,  it  can  be  ignored  thereafter.  Or  it  can  be  elimi- 
nated after  operator  identification  with  a cursor.  Consequently,  the  rate  at  which 
new  data  (with  new  false  alarms)  appears  is  the  rate  at  which  a totally  new  image 


1 

i 

t 

f 

- 


4.  Final  Report , Demonstration  of  Westinghouse  Automatic  Cueing  Techniques  Using 
NVL  Images  (U),  Westinghouse  Defense  and  Electronic  Systems  Center,  Document 
No.  9780,  May  1976. 

5.  Tisdale,  G.E.,  Automatic  Cueing  of  Reconnaissance  Imagery  - Performance  vs.  Cost, 
paper  presented  at  the  Symposium  on  Computer  Image  Processing  and  Recognition, 
University  of  Missouri,  Columbia,  August  24  - 26,  1972. 
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is  in  the  field  of  view.  Regardless  of  whether  the  application  involves  heli- 
copters, high  speed  aircraft,  missiles  or  RPV's,  the  geometry  shown  below  is 
approximately  representative.  The  platform  is  at  altitude  h,  looking  out  at  a 


slant  range  R,  and  having  an  angular  field  of  view  9,  and  the  area 
observed  is  d feet  long.  When  the  platform  has  travelled  forward  by  d feet, 
a whole  new  image  will  be  viewed.  If  the  platform  has  a forward  speed  of  v, 
the  time  between  new  frames  is 


T = h/v  {tan[cos_1(h/R)  + 0/2]  - tan[cos  1(h/R)  - 0/2]} 


and  the  effective  false  alarm  rate  becomes 


FAReflfl  = P x — false  alarms /sec. 


A graph  of  T is  shown  in  Figure  2-2;  the  kind  of  mission  considered  will  place 
limits  on  h,  r,  v,  and  T. 


Mr.  Donald  Looft,  Deputy  Director  of  DARPA,  at  the  sixth  annual  Automatic 
Imagery  Pattern  Recognition  Sumposium  of  the  EIA,  in  June  of  1976,  suggested 
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several  scenarios  during  his  banquet  speech.  The  particular  examples  extracted 
for  this  project  cover  helicopters  and  high  speed  aircraft  with  airborne  infrared 
imaging  systems. 


"Now  visualize  a likely  scenario  for  the  one-man  pilot  observer.  He  must 
direct  the  sensor  and  visually  scan  his  display  for  targets  which  most  assuredly 
are  concealed  by  terrain  or  background  clutter.  He  must  fly  low  and  as  fast  as 
possible  to  minimize  exposure  to  radar  controlled  weapons,  say  in  the  helicopter 
case,  speeds  of  60  to  80  knots  at  altitudes  of  100  to  200  feet  are  typical. 

In  the  case  of  attack  aircraft T speeds  of  400  to  500  knots  at  altitudes  of  two 
to  three  kms.  (6560  - 9840  feet)  are  normal  flight  profiles,  though  the  latter 
can  also  be  "down  on  the  deck".  At  the  same  time  the  man  or  the  crew  is  per- 
forming these  flight  maneuvers  which  are  "hairy"  in  broad  daylight,  let  alone 
darkness,  he  must  monitor  his  aircraft  instruments,  he  must  be  wary  of  terrain 
obstacles  and  he  may  well  be  under  enemy  fire.  Ideally,  the  pilot  wants  to  acquire 
and  engage  targets  on  a single  pass.  If  he  must  take  several  passes,  his  surviva- 
bility is  greatly  reduced.  If  he  has  been  directed  to  a probable  target  area  and 
is  able  to  take  advantage  of  terrain  masks  he  must  "pop  up",  make  his  observations, 
acquire  and  engage  targets  in  times  like  15  or  20  seconds.  He  would  always  like 
to  acquire  and  engage  enemy  targets  at  sufficient  standoff  distance  to  be  out  of 
range  of  radar  controlled  enemy  air  defense  weapons.  This  means  he  is  usually  at 
or  near  to  threshold  recognition  range  of  the  sensor." 

From  these  scenarios,  it  is  possible  to  discern  several  bounds  on  the  effec- 
tive false  alarm  rate.  The  pilot  observer  who  is  performing  difficult  maneuvers, 
monitoring  his  platform  instruments,  and  avoiding  terrain  obstacles  is  interested 
in  a false  alarm  rate  in  the  order  of  10' s of  seconds  per  alarm,  or  higher.  Given 
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a helicopter  at  200  feet,  © = 5°>  v = 118  nmi/hr.  (Figure  2-2)  and  looking  ahead  at 
a range  of  say  2300  feet,  the  time  between  new  information  frames  is  15  seconds. 


For  a FAReflf  = 1/20,  = 15/20  = .75.  An  even  better  FAReff  is  1/100,  so  that 


Pfa  " 


For  aircraft  speeds  of  5*  the  helicopter  speed  and  an  altitude  of  6000  feet 


and  slant  range  of  12000  feet,  the  time  between  frames  T = 1/5  T1QOm  = 7 seconds 


and  the  above  numbers  are  reduced  by  7/15. 


The  above  has  been  concerned  with  a pilot  observer  who  cannot  be  occupied 
full  time  with  a display.  In  this  case  it  has  been  suggested  that  an  effective 
false  alarm  rate  of  one  per  10  to  20  seconds  is  appropriate,  although  further 
examination  by  human  factors  specialists  is  clearly  in  order.  Another  case  is 
that  of  a full  time  observer,  such  as  an  RPV  operator.  Here,  it  may  be  possible 
to  tolerate  a higher  false  alarm  rate,  perhaps  up  to  one  per  second,  since  no 
additional  tasks  are  involved. 


2.2.3  Speed  of  Operation 


The  speed  of  operation  of  the  cueing  system  depends  on  the  number  of  sensor 
frames  to  be  processed  per  second.  The  frame  rate  and  frame  size  determines  the 
sample  or  pixel  rate. 


To  obtain  design  goals  for  frame  rates  and  pixel  rates , it  seems  reasonable 
to  start  with  something  familiar  that  is  likely  to  be  applied  to  automatic  cueing 
systems:  namely,  the  standard  TV  format.  With  525  horizontal  lines  and  assuming 

635  pixels  per  line  (which  would  correspond  to  high  quality  broadcast  TV),  there 
are  333,375  pixels  per  frame.  This  number  may  be  high  because  it  represents  the 
picture  sent  to  a home  receiver  by  commercial  TV.  The  home  receiver  projects  a 
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picture,  which  due  to  interference,  etc.  is  more  like  525  lines  by  1*90  pixels 
or  257,250  pixels  per  frame.  The  TV  frhme  rate  is  30  per  second,  which  results 
in  sample  rates  of  10  megapixels  and  7.7  megapixels  respectively,  if  each  frame 
is  processed. 

What  effect  does  the  automatic  cueing  process  have  on  frame  rates?  One 
factor  is  the  slew  rate  of  the  sensor;  assume,  for  example,  that  the  sensor  is 
responsible  for  120  degrees  of  azimuth  coverage  and  the  platform  is  a helicopter 
in  the  "pop  up"  mode.  If  the  allowable  reaction  time  is  20  seconds^,  then  the 
field  must  be  scanned  in,  say,  8 seconds  for  a slew  rate  of  15  degrees /second 
and  the  human  observer  has  another  12  seconds  to  verify  and  respond.  Assume 
a F0V  of  5 degrees,  then  2k  processing  frames  are  necessary  for  azimuth  coverage. 
For  an  eight  second  time  interval,  this  reduces  to  three  frames  per  second  or 
every  tenth  TV  frame.  The  pixel  rate  in  this  case  becomes  1 megapixel  per  second 
to  0.77  Megapixels  per  second.  If  the  field  of  view  is  increased  to,  say,  10 
degrees,  the  frame  rate  becomes  1.5  frames  per  second  and  a rate  of  0.5  megapixels/ 
second.  Clearly,  there  is  a trade  off  between  slew  rate,  pixel  rate,  and  field 
of  view.  The  pixel  rate,  algorithm  complexity,  and  machine  speed  are  inter- 
dependent. Machine  speed  may  in  turn  be  increased  by  parallel  processing  which 
implies  duplicate  circuitry  and  increased  expense.  The  design  goal  is  difficult 
to  define  precisely  without  an  operations  analysis  of  each  mission.  Prime  factors 
to  be  considered  are  the  coverage  desired,  and  the  amount  of  time  necessary  to 
complete  that  coverage.  In  spite  of  the  cursory  nature  of  the  above  analysis,  it 
does  indicate  a level  of  magnitude  for  frame  rates  and  pixel  rates . 


6.  Looft,  D.J.,  Image  Understanding  — A Perspective  of  POD  Needs,  banquet 

speech  presented  at  the  Sixth  Annual  AIPR  Symposium,  College  Park,  Maryland, 
June  1,  1976. 


There  are  several  other  factors  which  may  be  presented  in  support  of  frame 
rates  lover  than  30  per  second.  For  example,  once  the  system  has  detected  a tar- 
get, it  is  available  to  the  operator  to  be  dealt  with;  it  is  no  longer  necessary 
to  reacquire  at  30  frames /second.  Further,  as  discussed  in  Par.  2.2.2,  some  work 
indicates  that  the  process  is  not  noise  dependent;  i.e.,  detections  and  false 
alarms  do  not  change  with  repeated  looks  at  the  scene.  This  implies  that  the 
processing  frame  rate  should  be  a function  of  the  rate  of  change  of  the  scene.  From 
Figure  2.2.,  this  may  vary  from  2 to  10  seconds  which  would  imply  aprocessing  rate  of 

0.5  to  0.1  frames  per  second. 

2.3  PHYSICAL  CHARACTERISTICS 

The  physical  characteristics  of  the  automatic  cueing  system  include  the 
allowable  size  and  weight,  and  the  power  consumption. 

The  size  and  weight  of  the  cueing  system  are  constrained  by  two  limits.  One 
limit  is  the  allowable  space  and  weight  aboard  the  vehicle.  The  other  limit  is 
the  minimum  amount  of  hardware  required  to  perform  acceptable  cueing.  Hopefully, 
the  latter  value  is  less  than  the  former.  The  advent  of  medium  scale  integration 
(MSI)  techniques  and  large  scale  integration  (LSI)  techniques,  coupled  with  special 
chips,  will  contribute  substantially  to  a significant  reduction  in  both  size  and 
weight.  Examples  of  this  are  commercially  available  1 megabit  CCD  serial  memories 
on  a 9 x 15  inch  board.  Another  example  is  the  field  programmable  logic  array 
(FPLA)  which  places  24  inverters,  48  AND  gates,  and  8 OR  gates  on  a 24  pin  chip. 
Further,  special  chips  are  useful  in  that  shift  registers  controlling 
data  flow  numbering  perhaps  25  chips  might  become  a single  chip  representing 
a special  purpose  stack  register.  Of  course,  bare  chip  packaging  would 
further  reduce  size  and  weight  figures.  With 
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regard  to  hardware  implemenation,  it  seems  not  at  all  unreasonable  to  expect 
the  system  to  be  in  the  neighborhood  of  0.5  to  1.5  cubic  feet,  excluding  displays, 
and  to  weigh  a nominal  15  to  30  lbs.  including  power  supply. 

The  requirements  of  a specific  platform  are,  as  mentioned,  another  conside- 
ration. The  aircraft  and  helicopter  could  probably  accommodate  the  upper  end  of 
the  hardware  estimates,  i.e.,  1.5  cubic  feet  and  30  lbs.;  however .missile  installa- 
tion would  produce  more  stringent  requirements,  such  as  0.5  cubic  feet  and  15  lbs. 

Power  design  goals,  assuming  the  automatic  cueing  system  has  its  own  power 
supply, can  be  roughly  bounded.  Certainly  even  for  the  case  of  aircraft  and  heli- 
copters, 1000  watts  is  too  high;  in  fact,  less  than  one  fifth  that  number  is 
probably  more  appropriate  with  a still  lower  power  requriement  associated  with 
the  missile.  It  appears  that  numbers  in  the  neighborhood  of  200  - 300  watts  are 
reasonable.  It  must  be  cautioned  that  the  size,  weight  and  power  quantities  are 
considered  to  be  design  goals  and  not  hard  requirements;  they  represent  a first 
attempt  to  define  a set  of  acceptable  numbers  with  which  the  users  may  not  be 
completely  satisfied,  but  which  will  be  of  some  interest.  Clearly,  interest  will 
increase  by  the  amount  that  the  design  goals  are  met  and  exceeded. 

2.1+  ALLOWABLE  COST 

The  allowable  cost  of  an  automatic  cueing  system  will  ultimately  relate  to 
its  dollar  value.  Determination  of  this  value  will  depend  on  the  increase  in  the 
overall  system  performance  resulting  from  its  use. 


Acquisition  of  targets  is  valuable  primarily  because  it  results  in  action 
of  some  kind.  The  cueing  must  take  place  in  a system  that  can  benefit  from 


increased  information.  Improvements  in  weapon  delivery  systems,  recently  de- 
scribed in  the  press,  should,  in  turn,  require  more  effective  target  acquisition. 

Suppose  the  cueing  machine  could  result  in  doubling  the  effective  action  of 
a particular  mission.  Then,  it  has  doubled  the  mission  value.  To  put  it  another 
way,  half  as  many  missions  would  be  required  to  achieve  the  same  result. 

Placing  a dollar  value  on  the  results  of  a military  mission  appears  futile. 
However,  it  seems  reasonable  to  assume  that  on  the  average,  the  value  exceeds 
the  cost;  otherwise,  such  missions  would  not  be  attempted.  The  mission  cost  in- 
cludes the  mission  operating  expenses,  and  the  appropriate  depreciation  of  the 
total  aircraft  system. 

The  value  of  the  cueing  machine  is,  therefore: 

Mission  Cost  x Total  Missions  Flown  x Effective  Increase  in  Performance 

i 

If  we  assume  that  the  automatic  cueing  machine  will  perform  for  the  life 
of  the  aircraft,  then  the  first  two  factors  above  can  be  replaced  by  the  total 
aircraft  cost  plus  the  expense  of  all  operations.  If  the  effective  increase  in 

[ 

| performance  caused  by  the  cueing  machine  were,  in  fact,  doubled,  then  clearly, 

its  value  would  be  very  large.  Since  its  cost  will  be  only  a small  fraction  of 

I this  amount,  for  any  significant  increase  in  performance,  it  will  be  highly  cost 

I effective. 

' 

In  general,  this  line  of  reasoning  can  be  carried  a step  further  to  infer 
the  optimum  level  of  complexity  of  the  cueing  machine.  The  solid  curve  in 

I 

Figure  2-3  represents  the  machine  performance  as  a function  of  circuitry  complexity. 
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from  A to  B is  experienced  as  additional  equipment  is  put  to  good  use.  Beyond 
B,  however,  saturation  occurs,  because  no  amount  of  equipment  will  produce 
performance  above  100$.  The  dashed  line  represents  the  condition  stated  pre- 
viously, that  the  value  assigned  to  the  machine  is  proportional  to  mission 
performance  improvement.  Since  it  touches  the  smooth  curve  only  at  B,  it 
describes  a mission  that  can  make  cost  effective  use  of  the  automatic  cueing 
machine  only  if  its  complexity  (point  C)  provides  performance  approaching  the 
maximum.  Value  lines  above  this  will  not  result  in  cost  effective  operations 
while  lines  below  it  will  offer  solutions  over  a wide  range  of  complexities. 

However,  the  most  effective  operating  point  will  remain  in  the  vicinity  of  B. 

Although  it  may  be  argued  quite  convincingly  that  the  cueing  system  could  be 
priced  in  accordance  with  its  value,  as  a practical  matter  the  upper  price  limit 
will  be  dictated  by  other  considerations.  There  is  a strong  tendency  at  present 
to  provide  a fixed  amount  of  money  for  an  overall  military  system  (design  -to -cost 
philosophy),  such  as  an  aircraft  or  helicopter.  Accordingly,  regardless  of  value, 
the  cueing  system  must  compete  for  acceptance  with  many  other  important  items , 
such  as  sensors  and  navigation  devices.  Because  the  cueing  function  is  a recent 
development,  which  is  not  fully  understood  or  appreciated,  it  will  suffer  in  the 
competition  for  funding.  The  real  funding  limit  will  depend  on  the  circumstrances 
in  each  application. 


t 
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Performance 


Figure  2-3.  Performance  vs  Machine  Complexity  or  Mission  Value 


3.0  HARDWARE  CONSTRAINTS 


This  section  discusses  constraints  placed  upon  the  algorithms  by  the 
hardware;  certain  mathematical  functions,  logic,  and  memory  structures  have 
been  found  to  be  difficult,  in  terms  of  chip  count  and  processing  time, 
to  implement  and  should  be  avoided  by  the  analyst.  Unfortunately,  all  too 
often,  the  hardware  designer  is  given  a completed  and  checked  out  algorithm  to 
implement.  To  change  the  algorithm  structure  at  that  point  would  require  that 
the  analyst  redesign  the  algorithm  and  repeat  the  checkout  process  which  is 
usually  not  budgeted.  This  section  is  written  to  avoid  that  dilemma. 

The  system  design  goal  section  has  suggested  that  an  automatic  cueing  system 
should  be  capable  of  performing  rates  of  roughly  1 megapixel/sec.  This  implies 
that  there  is  a 1000  nanosecond  time  interval  between  pixels,  which  becomes  a 
rough  bound  on  machine  speed.  To  complete  an  algorithm  within  a frame  time,  it 
probably  must  be  repeated  for  each  pixel  which  means  that  the  algorithm  must  be 
completed  on  a per  pixel  basis  in  1000  nanoseconds. 


The  use  of  microprocessors  is  generally  ruled  out  with  a 1000  nanosecond 
processing  time  because  the  faster  bipolar  microprocessors  have  instruction 
cycle  times  (through  the  arithmetic  logic  unit  (ALU)  and  back)  of  200  nano- 
seconds. In  addition,  time  is  involved  in  fetching  instructions,  decoding 
instructions,  putting  data  into  the  RAM  portion  of  the  ALU,  and  finally,  executing. 
Power  requirements  limit  the  microprocessors  to  transistor-transistor  logic  (T  L) 
rather  than  emitter-coupled  logic  (ECL)  which,  of  course,  is  much  faster.  So 
the  hardware  thrust  seems  to  lie  in  the  direction  of  clock-controlled  sequential 
logic.  Again,  this  is  not  limited  to  pipeline  arrangements  because  parallel 
processing  has  obvious  advantages  in  certain  situations. 

Clearly,  the  kinds  of  times  involved  prohibit  other  than  very  simple  arith- 
metic such  as  sums  and  differences.  Numerical  integration,  quadratic  smoothing, 
etc.,  and  techniques  normally  associated  with  larger  machines,  although  not 
impossible  to  implement,  will  be  costly  to  implement  and  less  likely  to  succeed. 
Sums  and  differences  will  probably  be  performed  with  table  look-up  techniques  if 
digital  processing  is  desirable. 

Another  constraint  acts  on  the  kinds  of  numbers  involved.  In  general,  the 
fewer  the  number  of  bits  to  represent  a number,  the  better.  This  is  advantageous 
on  two  counts.  First,  the  devices  can  be  much  simpler.  Second,  the  devices  can 
be  faster. 

The  1000  nanosecond  constraint  puts  tight  limits  on  the  types  of  algorithms 
which  one  can  consider  applying  to  the  whole  image  or  major  parts  of  it. 

The  problems  of  memory  access  are  typical.  In  general,  the  memory  content 
is  shifting  at  the  rate  the  image  is  scanned.  If  an  algorithm  calls  for  a change 


in  a variable  memory  location  (pixel)  based  on  algorithm  results,  then  random 
access  must  be  obtained  to  a shifting  memory  in  a matter  of  100  nanoseconds  or 
so.  The  assumption,  here,  is  that  most  of  the  allowable  1000  nanoseconds  is 
consumed  by  the  algorithm  execution.  If  a small  portion  of  the  image  is 
affected,  then  a RAM  in  parallel  with  a large  shift  register  (1*000  word  CCD, 
for  example)  can  accomodate  the  memory  access.  However,  if  the  portion  affected 
is  large,  the  memory  now  must  be  composed  of  a large  number  of  RAMs  or  a large 
shift  register  with  intolerably  long  access  times.  In  the  former  case,  the 
chip  count  is  substantially  increased;  in  the  latter  case,  the  megapixel  rate 
may  not  be  achieved  since  access  times  are  in  microseconds.  Some  other  guide 
lines  are  listed  in  the  next  several  paragraphs. 

In  general,  the  algorithm  should  be  as  repetitive  as  possible  in  that  it 
does  the  same  operation  of  every  point  in  the  picture  (like  a simple  FORTRAN  DO 
loop  without  conditional  statements). 

In  general  the  algorithm  should  require  a minimal  (e.g.,  l)  number  of 
"passes"  over  the  image  to  minimize  storage,  chip  count,  and  time.  An  example 
would  be  algorithms  which  do  not  include  edge  and  region  growing  approaches . 

In  general,  the  use  of  trigonometric  functions  should  be  limited, especially 
in  analog  and  CCD  implementations . 


h . 0 HARDWARE  IMPLEMENTATION 


The  algorithms  received  fall  into  two  classes  which  can  be  roughly  classified 
as  thresholding  and  noise  filtering  algorithms.  The  threshold  algorithm  is  im- 
plemented with  digital  circuitry  and  the  overall  data  flow  is  examined;  analog 
implementation  is  considered  for  the  same  algorithm.  Parallel  processing  and 
recursive  formation  of  histograms,  averages,  and  threshold  comparisons  are  also 
discussed.  The  noise  filtering  algorithm  is  implemented  with  digital  techniques 
since  it  is  essentially  a binary  algorithm  in  its  conception.  A discussion  of 
alternate  hardware  approaches  to  these  algorithms  follows  with  some  indications 
of  future  directions.  This  section  concludes  with  a discussion  of  CCD  techniques 
and  fabrications  applicable  to  these  algorithms  for  possible  on-focal-plane  pro- 
cessing. It  should  be  emphasized  that  the  intent  of  this  section  is  to  implement 
the  specific  algorithms  and  not  to  provide  a general  discussion  of  LSI  techniques 
which  are  available  in  the  open  literature.  Because  we  were  able  to  start  Phase 
II  work  during  Phase  I,  it  was  felt  that  direct  implementation  was  more  beneficial 
to  our  colleagues  of  the  Univeristy  of  Maryland  and,  ultimately,  to  the  project. 

4.1  THRESHOLD  ALGORITHM 

There  are  four  variations  of  the  thresholding  concept  , but  central  to  all  of 
them  is  the  sequence  of  calculating  the  gradient  and  forming  a histogram  in  order 
to  find  the  gradient  value  corresponding  to,  say,  80%  of  the  population  in  a frame. 

Then,  using  this  amplitude  , the  lower  80%  of  the  gradient  population  is 
discarded.  The  remaining  pixels  also  have  a gray  level  associated  with  them;  the 
average  amplitude  is  computed  and  used  to  threshold  and  discard  those  amplitude 
values  below  the  average.  The  particular  gradient  threshold  technique  analyzed 
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Consider  the  operator,  p(X)  = max{|A  - B|,  |C  - D|};  each  term  is  shown  in 
Figure  1+-1  with  the  composite. 
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Figure  4-1.  A,  B,  C,  D and  Composite 


a^  is  a pixel  element  in  A (2  x 2) 

b is  a pixel  element  in  B (2  x 2) 
J 

c is  a pixel  element  in  C (2  x 2) 

K 

d is  a pixel  element  in  D (2  x 2) 
in 


A,  B,  C,  and  D each  form  a 2 x 2 matrix;  A,  B,  C,  and  D each  represent  the  sum 
of  all  four  elements;  each  element  corresponds  to  a picture  element.  Since  the 
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value  of  each  picture  element  is  a gray  level  with  a value  0 < e 63,  the  sums 
are  formed  with  positive  quantities  which  may  be  analog  or  digital.  These  sums 
may  be  formed  serially  by  scanning  up  or  down  the  frame  or  in  parallel  as  in  an 
array.  Serial  formation  is  discussed  first.  A could  be  formed  by  scanning 
across  two  lines  and  placing  the  sum  in  an  appropriate  location.  could  be 
placed  in  the  upper  left  pixel  location  for  A^,  or  it  could  just  have  easily 
have  been  placed  at  the  X position  as  in  Figure  h-2. 
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Figure  h-2.  Alternate  Placements  of  the  Sum,  A1 


In  general,  it  would  be  convenient  to  place  the  sums  A,  B,  C,  and  D such  that 
indexing  systems  are  simple. 


Figure  U-3(a) shows  how  A could  be  formed  by  conventional  circulating  CCD 
shift  registers  and  some  parallel  in-parallel  out  shift  registers. 

CCO  SHIFT  REGISTER 


'£ 
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Expanding  on  this  idea.  Figure  4-3(b)  shows  the  formation  of  A,  B,  C,  D. 


Figure  4-3(b).  Formation  of  A,  B,  C,  D with  Registers. 


where : 


A e12  + ei3  + e22  + e23 

® = e32  + e33  + eh2  + ei+3 

6 = e21  + e22  + e3]  + e32 

^ = e23  + S2k  + e33  + S3i+ 

e = picture  element  in  row  i and  column  j, 
1 J 


Assume  that  each  box  in  the  4 x 4 array  is  a parallel-in-parallel-out  shift 
register  with  a dual  output  which  allows  the  output  of  each  shift  register  to  be 
sent  both  to  the  next  shift  register  and  to  an  arithmetic  logic  unit  (ALU)  where 
p(X)  is  computed.  An  expanded  view  of  the  4 * 4 array  is  shown  in  Figure  4-4. 
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Figure  4-4.  Expanded  View  of  4 x 1*  Array 

When  the  gray  level  amplitude  located  at  e22  is  shifted  to  the  e21  position, 
it  is  also  sent  to  the  ALU  where  a new  p(X)  is  calculated.  The  rate  at  which 
the  frame  is  scanned  and  p(X)  is  formed  is  limited  by  the  shift  rate  of  the  shift 
registers,  which  could  be  multiplexed  to  produce  higher  rates,  and  the  speed  at 
which  p(X)  is  formed  by  the  ALU.  Recall  that  the  present  analysis  is  only  con- 
cerned with  a serial  scan  of  the  frame  and  the  option  to  process  data  by  analog 
or  digital  is  now  addressed. 

4.1.1  Digital  Implementation 

Assume  the  pixels  have  been  digitized  and  are  represented  by  five  bits, 

0 <_  gray  level  <_  31.  The  ALU  is  formed  in  Figure  4-5. 
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Figure  4-5 . Arithmetic  Logic  Unit  • co»mmto« 
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The  quantities  e^g,  e^,  •••  are  fi-ve  tits  long  which  together  form  a 
ten-bit  address;  there  are  1024  combinations  of  address  which  calls  for  a PROM 
which  is  1024  x (at  least)  five  bits  wide.  The  sum  is  stored  at  each  address 
and  read  out.  The  second  stage  of  PROM's,  9 - 12,  has  the  same  characteristics 
as  do  PROM's  13  and  l4.  Typically,  these  PROM's  have  a propagation  delay  of 
90  nanoseconds  for  Schottky  and  low  power  Schottky.  Emitter  coupled  logic,  ECL, 
will  not  be  considered  because  of  excessive  power  requirements.  Assuming  a 
settling  time  in  the  shift  registers  of  30  nanoseconds,  and  a comparator  pro- 
pagation time  of  60  nanoseconds,  the  total  time  to  produce  p(X)  is  360  nanoseconds. 
This  time  could  be  reduced  by  substituting  adder  chips  for  the  PROM's,  to  240 
nanoseconds,  but  the  number  of  chips  would  have  increased  from  15  to  at  least  30. 
Another  way  to  reduce  propagation  time  from  e^  to  p(X)  is  to  employ  a single  PROM 
stage  to  produce  A,  B,  C,  and  D each.  To  form  A,  the  address  is  20  bits  wide 
composed  of  2±2  e13  e22  e23,  or  four  PROM's  wide  and  32^1024  PROM's  long 
which  is  a substantial  increase  in  chips  for  a reduction  of  90  nanoseconds. 

Thus,  perhaps  an  optimum  combination  of  chips  and  speed  is  15  chips  and  360  seconds. 

Another  consideration  is  the  forming  of  a histogram  composed  of  p(X.  )'s. 

^ J 
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The  operator  pU^)  has  a range  of  values  0 < P(xij)  1 31  and  a five-hit 


binary  to  decimal  decoder  would  place  the  p(X  ) value  in  one  of  32  slots.  A 
counter  and  register  for  each  decimal  line  would  accumulate  the  population  in 
each  slot,  as  shown  in  Figure  4-6. 


P (X.) 
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Figure  4-6.  Histogram  Formation 

Over  one  frame,  the  number  of  pixels  is  a known  quantity  and  the  number  of 

pixels  corresponding  to  80  percent  of  the  pixels  is  also  known,  say  C^,  so  that 

the  registers  0-31  need  to  be  summed  sequentially  until  C.^  is  reached.  A counter 

would  track  the  registers  to  determine  which  register  was  being  summed  when  C1  was 

reached.  This  value  p (X^)  then  serves  as  a threshold  which  is  compared  to  all 

the  p(X ,'s)  in  a frame.  Before  and  after  comparison,  the  frame  would  look  like 
^ J 

Figure  4-7. 
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Figure  4-7.  Before  and  After  Comparison 
4. 1.1.1  Overall  Data  Flov 

It  is  appropriate  to  review  the  digital  circuitry  derived  so  far  and  put  it 

on  one  piece  of  paper.  Note  that  after  the  computation  of  each  p(X..),  it  is  stored 

1 J 

in  a set  of  CCD  shift  registers  at  the  left.  When  p(Xcl)  is  obtained,  all  p(X^j)'s 
in  the  frame  have  been  stored  in  the  p(X^j)  registers.  Now,  each  p(X^)  value 
is  shifted  out  and  compared  with  p(X^)  in  a comparator.  At  the  same  time  that 
the  operations  to  obtain  p(Xcl)  are  being  performed,  the  gray  level  amplitudes 
are  being  stored  in  another  set  of  shift  registers,  the  e^j  registers.  If 
p(XCi)  <_p(Xij),  then  e^  is  stored  in  the  e^  registers  at  position  ij , if 
p(Xni ) > p(X. .)  then  0 is  stored  at  position  ij . At  the  same  time,  the  value 
eij's  are  being  cumulated  and  counted  to  obtain  an  average  e.  When  e has  been 
obtained,  the  remaining  e 's  are  then  compared  to  e,  to  eliminate  another  group 
of  data. 
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The  entire  frame  must  be  cycled  through  to  obtain  p(X  ),  and  recycled 

v/X 

to  compare  p(X^)  with  p(X^j ) to  obtain  a first  filtering,  this  simultaneously 
gives  F.  The  frame  must  be  cycled  again  to  filter  the  amplitude  data  based  on 
e.  This  gives  an  idea  of  the  overall  flow  of  data.  One  should  note  that  in 
the  left  side  of  Figure  4-8,  there  are  simple  adds  and  compares.  However,  in 
the  right  hand  side,  there  is  more  time  taken  in  computation.  Consider  an 
analogue  implementation  of  the  same  algorithm. 

4.1.2  Analog  Implementation 

The  basic  tool  used  in  the  analog  analysis  is  an  operational  amplifier, 
with  a gain  of  unity,  which  comes  in  a module  of  dimensions  1 x 2 x .5  inches. 
The  settling  time  for  1%  accuracy  is  approximately  40  to  50  nanoseconds.  Either 
A,  B,  C,  or  D may  be  produced  by  a summing  amplifier  of  the  form,  shown 
Figure  4-9. 


Figure  4-9.  Summing  Amplifier 
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A = ~(a22  + a23  + a32  + a^),  where  R0  = \ = R2  = R3  = Rj* 


r 


and  this  amplifier,  because  of  multiple  inputs,  may  have  a settling  time  of  80 
nanoseconds.  Note  that  settling  time  is  the  same  as  the  propagation  time  in 
digital  circuitry.  Similarly,  B = -(b^2  + ^3  + ^52  + ^53^  an<^  ^our 
are  needed  to  obtain  A,  B,  C,  and  D.  Since  &22,  a23’  a32’  anti  a33  are  Posa‘t^ve 
quantities , as  are  all  elements,  A - B may  be  obtained  from  a differential  ampli- 
fier, (Figure  4-10),  or  a two  stage  invert  and  add.  EQ  = (B  - A)RQ/R  , where 

R0  = R1  » 


R0 


Figure  4-10.  Differential  Amplifier 

but 

A,  B < 0 from  the  previous  amplifier,  therefore 
EQ  = (A  - B)  , 

and  Eq  will  be  positive  or  negative  depending  on  the  relative  magnitudes  of  A and  B. 
Before  considering  this,  it  is  worth  noting  that  A - B may  be  obtained  in  one  step, 


Figure  i*-10. 


A - B = E, 
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with  a settling  time  in  excess  of  100  nanoseconds.  For  comparison  of  j 
and  |C  - D|,  it  is  necessary  to  obtain  the  absolute  value  of  both  A - 
with  the  following  type  of  circuit.  Figure  U-12. 


2B  +V  2R 
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Figure  k-12.  Absolute  Value  Circuit 


at 


If  A - B = -V,  then  the  circuit  produces  2V  - V = V,  if  A - B = +V  the 
circuit  produces  V.  The  propagation  time  is  roughly  130  nanoseconds.  The  output 
is  fed  to  a comparator  which  contains  a switch  allowing  p(x^)  = | A — B | or  | C — D | . 
Thus  far  the  propagation  time  is  of  the  order  of  310  nanoseconds  which  is  close 
to  the  digital  result.  Forming  the  histogram  is  considered  next. 

Here  a bank  of  voltage  comparators  can  be  selected  with  a switch  which  feeds 
a counter  associated  with  each  voltage  comparator  and  the  number  of  times  a voltage 
comparator  is  triggered  can  be  tabulated  and  the  histogram  formed.  The  counters 
are  summed  and  the  Q0%  value  of  p(x.  ) selected.  It  seems  possible  to  accomplish 
the  entire  process  in  less  than  about  1*00  nanoseconds.  Let  us  continue  now  and 
examine,  briefly,  the  possibilities  of  parallel  processing,  not  specifying  whether 
it  is  analog  or  digital. 

4.2  PARALLEL  PROCESSING 

Actually,  a direct  approach  is  to  divide  the  frame  into,  say,  horizontal 
slices  of  a third  of  a frame  in  the  vertical  dimension.  Constructing  the  histo- 
gram and  interfacing  problems  may  contribute  to  the  propagation  time  so  a more 
I conservative  approach  may  be  to  divide  the  frame  into  four  or  five  parts.  It 

might  be  suggested  that  the  price  has  increased,  and,  while  this  is  true, 
the  increments  are  small  because  a 1+096  x 1*  CCD  digital  shift  register  is  about 
$30  to  $1+0  in  small  quantities.  Pictorially,  Figure  1+-13  shows 


I 


Figure  4-13.  Parallel  Processing 

the  frame  divided  into  four  horizontal  sections.  Recall  that  the  P(xij),s 

f must  he  compared  with  p(X^)  , so  the  p(x^j)*s  must  be  saved.  Further  the 

amplitude  information  in  each  frame  must  also  be  saved,  so  that  additional 
memories  of  the  entire  frame  in  pU^)  and  eij  must  be  stored.  The  subsequent 
compares  of  p(Xcl)  vs.  p(x  ) and  e vs*  eij  are  limited  by  the  shift  rates  of 
the  registers  and  the  comparison  function  itself.  The  longest  task  seems  to 
be  deriving  pCx^).  Another  possibility  is  to  update  p(XCl)  and  e recursively 
and  also  perform  the  threshold  comparisons  at  the  same  time. 

4.3  NOISE  REGION  FILTERING 

The  Noise  Region  Filtering  Algorithm  acts  to  remove  noise  bursts  which  have 
survived  the  gradient  and  amplitude  thresholds,  but  are  small  in  area  and  isolated. 

i 
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Input  data  consists  of  an  entire  frame  whose  pixels  contain  either  0's  or  gray 


level  values  which  have  exceeded  past  thresholds.  For  this  test,  the  frame  should 


be  converted  to  0's  and  l's,  otherwise  the  filtering  will  be  more  difficult  to 


perform.  Suppose  that  when  the  amplitudes  were  compared  with  an  average  amplitude 


threshold  in  the  previous  test,  a "l"  was  stored  in  the  appropriate  pixel  location  j 


when  the  test  was  successful  and  a "0"  was  stored  when  the  test  was  unsuccessful. 


Thus,  the  input  data  for  the  Noise  Region  Filtering  Test  comprises  only  l's  and 


0's.  The  shrink  test  may  be  stated  as 


q'  (c) 


j~  1 if  e * 1 and  a + b + d + e = U; 


otherwise 


where  the  arrangement  of  pixels  a through  e is 


bed 


The  shrink  operation  is  equivalent  to  an  AND  gate  in  that  the  test  may  be  restated 


q'(c)  = 


1 if  a n b n c (1  d n e = 1 


0 otherwise. 


Since  the  data  is  already  binary,  and  each  pixel  is  represented  by  one  bit,  a 


five-bit  AND  gate  is  sufficient.  The  data  flow  could  consist  of  two  CCD  Bhift 


registers  holding  the  proper  rows  plus  several  smaller  shift  registers  to  hold 


the  immediate  a,  b,  c,  d,  e used  in  the  calculation.  The  "c"  shift  register 


would  also  contain  a dual  input  feature  such  that  the  result  of  the  AND  gate 


logic  could  be  entered  also.  The  block  diagram  appears  in  Figure  l+-lk.  To 
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Figure  U-lU.  Register  Block  Diagram 
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allow  symmetrical  timing,  four  additional  shift  registers  must  be  added  to  the 
first  and  third  rows  so  that  the  set  of  individual  registers  becomes  a 3 x 3, 
even  though  only  two  of  the  additional  registers  will  be  tapped.  The  contents 
of  registers  e12>  e21>  e22,  e^,  and  e^2  are  tapped  into  the  AND  gate  and  the 
results  placed  in  e22.  Figure  1+-15  sho^,-  the  data  flow  in 


which  e22  is  the  only  shift  register  which  requires  a dual  input;  all  the 
individual  registers  e^,  e^2>  •••  e33  8X6  1*1*  The  propagation  time  for 
the  process  will  approximate  50  nanoseconds  including  the  settling  time  for 
the  entry  of  e22  from  the  AND  gate.  To  produce  a second  shrink  test  a duplicate 
of  the  above  is  required. 

The  propagate  test  may  be  stated  as 

(lifc  = lora  + b + d + e>0; 

0 otherwise. 

This  test  states  that  q'(c)  = 1 if  any  of  a,  b,  c,  d,  e are  equal  to  1.  That 
is,  the  AND  gate  in  the  shrink  test  is  replaced  with  an  OR  gate.  Similarly, 
a two  step  propagate  requires  duplicate  OR  gates. 

U.U  ALTERNATE  HARDWARE  DIRECTIONS 

In  the  previous  sections,  the  purpose  was  to  implement  the  algorithms  in 
several  different  ways  without  regard  to  tradeoffs.  This  section  is  addressed 
to  the  tradeoffs  process,  conflicting  considerations,  and  future  directions. 

CCD  registers  were  employed  throughout  because  their  cost  per  bit  is  less 
than  conventional  shift  registers  and  they  can  obtain  the  necessary  shift  rates. 
However,  they  are  volatile;  i.e.,  they  lose  memory  when  there  is  a power  failure, 
whereas  the  Westinghouse  (BORAM)  Block  Oriented  Memory  does  not,  and  can  be  uied 
as  a shift  register.  However,  it  only  requires  a few  frames  to  restore  the  data 
base  with  comparable  shift  rates. 

Another  factor  is  the  array  of  smaller  shift  registers  used  to  hold 
the  appropriate  «1  »s,  which  are  then  sent  to  the  (ALU)  Arithmetic  Logic  Unit.  This 
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array  of  smaller  shift  registers  deletes  an  instruction  set 
needed  to  draw  the  appropriate  e^'s  from  the  larger  CCD 

shift  registers.  Another  approach  is  the  tapped  CCD  shift  register  with  non- 
destructive read  out,  developed  hy  Westinghouse,  which  would  delete  the  need 
for  a smaller  array.  However,  this  would  call  for  a special  chip. 

When  special  chips  are  considered,  it  is  possible,  for  example,  to  put  the 
ALU  of  the  threshold  algorithm  on  one  chip.  Whether  this  is  worthwhile  or  not 
will  depend  on  size  considerations  of  other  parts  of  the  hardware  implementation 
which  may  be  much  larger  than  the  ALU.  The  CCD  shift  register  arrays  necessary 
to  hold  a frame  of  e.'s  and  p(x.  )'s  are  now  discussed. 

1 J 1 J 

Assume  a frame  size  of  3^0,000  pixels  x 6 bits  per  pixel;  a CCD  array  of 
2,000,000  bits  capacity  is  needed,  A 1 megabit  array  is  commerically  available  of 
dimensions  128,000  words  x 8 bits  on  a 9 x 15  inch  board.  Special  designs  and  fabri- 
cations are  required,  and  will  be  considered  in  Phase  II,  to  reduce  the  word  size  to 
6 bits.  Assuming  for  the  moment  that  the  commercial  version  is  used  without  modification, 
three  boards  would  be  necessary  to  hold  e.  and  three  more  for  p(x. ,).  Further, 

1 J 1 J 

assuming  a spacing  between  boards  of  2 inches,  the  memory  volume  becomes  12  x 9 x 15 
inches  or  1620/1728  = 0.9I+  ft^  which  is  uncomfortably  large  at  this  early  design 
stage.  On  the  other  hand,  it  can  be  suggested  that  analog  memories  on  special 
chips  would  require  smaller  volumes.  In  fact,  assuming  a worst  case  wherein  even 
the  histogram  is  formed  with  analog  devices  which  might  require  some  50-odd  opera- 
tional amplifiers  yielding  a volume  of  perhaps  200  cubic  inches  including  support 
circuitry,  the  analog  implementation  may  still  be  smaller  than  a digital  imple- 
mentation. It  should  be  noted  that  the  first  cut  has  shown  the  speeds  of  analog 


and  digital  implementations  comparable  and  half  the  allowable  time  which  is 
a good  safety  margin  at  this  point.  Another  problem  which  occurs  in  analog 
implementation  is  signal  degradation  as  a function  of  the  shift  rate  and  length 

of  the  register,  e.g.,  a typical  example  of  a 100-stage  CCD  delay  line  with  an  00 

-k 

dB  dynamic  range  and  transfer  inefficiencies  of  10  at  1 to  2 mHz  clock  rates 
will  have  an  overall  signal  degradation  of  1%,  so  there  is  some  benefit  to  be 
gained  by  organizing  the  memory  to  reduce  the  number  of  shifts.  Such  organizations 
might  include  bringing  an  entire  line  into  memory  in  parallel  and  shifting  out  in 
series  or  multiplexing  out  to  reduce  shift  rates  also. 

' 

The  above  discussion  has  considered  the  threshold  algorithm  and  identified 
its  memory  requirement  as  the  more  interesting  among  other  challenges.  The  noise 
filtering  algorithm,  by  contrast,  seems  fairly  straightforward. 

Having  identified  the  memory  size  as  the  more  challenging  of  the  Phase  II 
hardware  problems,  the  next  section  is  a brief  description  of  some  of  the  West- 
inghouse  analog  techniques  and  devices  in  this  area  and  on- focal-plane  processing. 

It. 5 APPLICABLE  CCD  TECHNIQUES  AND  FABRICATION 

Any  signal  processing  system  that  involves  the  linear  transformation  of 
analog  signals  such  as  correlation,  discrete  Fourier  transformation  (DFT),  filter 
banks,  matched  filters,  multiplexing,  array  scanning,  orthogonal  scan  transfor- 
mation, time  base  translation,  etc,  can  be  realized  with  combinations  of  CCD 
basic  building  blocks.  In  discrete  analog  signal  processing  (DASP),  analog 
data  samples  are  stored,  transferred,  and  operated  upon  by  analog  means,  whereas 
in  conventional  digital  signal  processing  (DSP),  digital  or  quantized  samples 
are  handled  with  binary  logic.  A major  advantage  of  DSP  is  retained  by  DASP, 
namely  the  precise  transport  delay,  particularly  in  relation  to  coherent  signal 
processing.  The  dyanmlc  range  of  an  analog  bit  in  DASP  may  be  thought  of  as 
composed  of  6-dB  equivalent  DSP  digital  bits.  Thus,  a typical  example  of  100 
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Stage  (N  = 100)  CCD  delay  line  with  80-dB  dynamic  range  and  transfer  inefficiencies 
of  e ~ 10-4  at  l-to-2  MHz  clock  rates  will  have  an  overall  signal  degradation 
of  1 percent  (i.e.,  less  than  0.1-dB  insertion  loss)  without  the  need  of  A/D 


conversion. 


One-dimensional  basic  building  blocks  (linear  arrays)  may  be  classified 
according  to  the  characteristic  information  flow  patterns : 
o Serial  in/serial  out  (SI/SO) 
o Parallel  in/serial  out  (PI/SO) 
o Serial  in/parallel  out  (SI/PO) 
which  are  shown  in  Figure  4-l6. 

These  fundamental  linear  arrays  may  be  combined  to  form  area  arrays 
(2-dimensional  matrices)  with  increased  signal  processing  capabilities.  Table 
k-1  provides  a partial  listing  of  applications  for  these  basic  building  blocks. 


i.  si/so 


CCO  Wily  Line 
_ . N 


#.  PI/SO 


C.  SI/PO 


SI  With  Nondestructive  Sensing 
Tips  (Direct  or  with 
Multiplicative  Weighting) 


Figure  4-16.  CCD  Basic  Building  Blocks 
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TABLE  4-1. 


BASIC  INFORMATION  FLOW 

FUNCTIONAL  TABULATION  FOR  THE  DASP 
FAMILY  OF  DEVICES 


Basic  Information  Flow 


Serial  In;  Parallel  ln;  Serial  ln:  Parallel  Out  ( S l/PO) 

Array  Serial  Out  Serial  Out  - Nondestructive  Sensing  Taps 

Configuration  (SI/SO)  (PI/SO)  Unweighted  Weighted 


Linear 


Pure  Delay;  Time  Division  Beam  Focusing; 

Time  Base  Multiplexing;  Focus  Scanning 

Interchange  Array  Scanning  Multiple-Beam 

Forming; 

Beam  Steering 


Transversal  Filtering-. 
Correiation/Convoiution; 
Adaptive  Filtering-. 

Sampled  Data  Smoothing  or 
Interpolation  (Scan  Format 
Converter) 


2-d  Matrix 
(Areal 


Bulk 

Serpentine 

Analog 

Storage* 


T 

Corner  Turn  Discrete  Fourier  Transformers; 

(Orthogonal  Filter  Banks; 

Scan  Trans-  Multiple  Cross  Correlators 

formation) 


I I 

| Bulk  Serial-Parallel-Serial  (SPSI  Analog  Storage*  ; 


•As  For  Video  Refresh  Memories 
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Figure  4-17  illustrates  the  SI/SO  pin  numbers  on  the  PI/SO  and  SI/PO  CCD 
building  blocks  fabricated  at  Westinghouse  ATL,  since  the  latter  devices  may 
be  used  only  in  the  SI/SO  mode  with  pins  No.  3 through  No.  18.  The  PI/SO 
block  has  a 2-mil  wide  channel  in  the  basic  SI/SO  register,  while  the  SI/PO 
block  has  a 5-mil  wide  channel.  The  length  of  the  SI/SO  CCD  delay  line  is 
N = 44  stages  of  delay  which  was  designed  to  accommodate  either  20  parallel 
inputs  or  20  parallel  outputs,  each  input  or  output  separated  by  a stage  delay 


for  the  insertion  of  a reference  sample  between  successive  signal  and  reference 
samples.  All  of  the  gates  are  protected  against  static  charge  accumulation 
with  a series  resistance/Zener  diode  configuration.  The  substrate  pin  No.  10 
should  always  be  the  most  positive  and  the  maximum  substrate  voltage  is  +35V 
with  typical  operation  at  +20V.  CCD  analog  shift  registers  have  been  operated 
with  substrate  voltages  of  +8V;  however,  since  the  electrode  lengths  (i.e., 
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Figure  U-17.  PI/SO  and  SI/PO  CCD  Basic  Building  Blocks 
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polysilicon  electrode  length  = 12  ib,  aluminum  electrode  length  = 8 pm)  are 
not  minimum  size,  the  transfer  efficiency  improves  with  larger  clock  voltages 
due  to  the  increase  in  electric  fringe  field  beneath  the  electrodes. 

4.5.1  Serial  In/Serial  Out  (SI/SO) 

The  SI/SO  block  is  a simple  CCD  analog  shift-register  with  the  characteris- 
tics discussed  in  paragraph  4.7.  In  a linear  array  configuration,  the  SI/SO 
block  provides  analog  signal  delay  with  the  ability  to  provide  time  base 
translation.  A typical  dynamic  range  for  present-day  SI/SO  blocks  is  80  dB 
with  0.3  percent  linearity  and  clock  frequencies  from  1 kHz  - 1.0  MHz  for  a 
64  analog  bit  delay  line.  The  clock  requirement  may  vary  from  device  to  device 
with  voltages  varying  from  TTL  to  MOS  compatible.  In  general,  MOS-type  voltage 
swings  we  needed  to  obtain  dynamic  range  and  frequency  response.  The  capaci- 

p 

tance  loading  for  the  drivers  is  typically  0.2  pF/mil  of  active  clock  electrode, 

2 

e.g.,  clock  electrode  areas  of  1.5  mil  will  have  0.3  pF  capacitance.  Thus, 
a 64  bit  delay  line  will  offer  a loading  of  - 20  pF/ driver.  In  general,  CCD 
structures  have  not  been  built  with  interface/buffer  circuits  on  the  chip 
because  of  the  advanced  development  nature  of  the  work;  however,  CCD  chips  can 
be  fabricated  with  MOS,  CMOS,  or  bipolar  interface  circuits.  In  order  to  test 
SI/SO  blocks  without  on-chip  buffer  circuits,  a so-called  "open  collector" 
driver  may  be  employed.  This  driver  is  relatively  inexpensive  and  provides 
clock  voltage  swings  of  30V  up  to  2-MHz  clock  frequencies.  Clock  shaping  may 
be  accomplished  if  desired  by  the  use  of  a series  resistor,  which  also  protects 
the  drivers  in  the  pull-down  transient.  A CCD  chip  should  have  protective 
resistor  diode  combinations,  similar  to  MOS-type  circuits,  to  limit  the  dis- 
placement current  and  prevent  shorting  of  the  input  electrodes . 

For  analog  signal  processing,  as  discussed  in  the  introduction,  a desirable 
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feature  is  the  incorporation  of  an  ac  zero  reference  between  successive  signal 

samples , particularly  for  PI/SO  and  SI/PO  blocks.  In  addition,  sample  and  hold 

techniques  are  required  for  analog  signal  reconstruction  which  attenuates  the 

response  with  a S^n  ^ shape  factor.  The  input  to  a CCD  is  filtered  with 

TTf  /f 

. . c s 

sin  irAtf 

a c roll-off  where  At  is  the  sampling  window  aperture.  The  output 

it  Atfc 

after  sample  and  hold  requires  filtering  with  a low-pass  filter  with  ideal 
"brick-wall"  cut-off  at  fc/2,  the  Nyquist  limit.  Figure  4-l8  illustrates  an 
analog  output  swept  frequency  response  of  a CCD  SI/SO  block  with  sample/hold 
and  a 7 -pole  Butterworth  filter  (-3  dB  at  750  kHz)  to  filter  the  clock  and 
limit  the  aliasing  of  frequencies  higher  than  fc/2.  Thus,  in  a properly  de- 
signed CCD  analog  delay  line  the  frequency  response  is  limited  by  the  sample/ 
hold  and  low-pass  filter  characteristics.  Since  the  input  signal  to  the  CCD 
is  sampled,  an  input  prefilter  with  cutoff  at  f J 2 is  required  to  reduce 
aliasing  similar  to  the  output  filter. 


Figure  U-l8.  Frequency  Response  of  Sampled  CCD  Analog  Delay  Line;  100  kHz/Div. 
Horizontal,  f ■ 2.0  MHz;  Transfer  Inefficiency  e (fc  = 2.0  MHz)  = 

2 x 10“3  for  L = 12  pm  Electrodes;  Sample/Hold  and  Filter  ^ 

Responses  are  Included  in  the  Overall  Response 
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4.5.2  Parallel  In/Serial  Out  (PI/SO) 

The  PI/SO  "block  may  be  used  to  time  division  multiplex  a number  of  low 
data  signal  channels  into  a higher  data  rate  output  channel.  The  variations 
in  electrical  input  may  be  minimized  with  the  use  of  a stabilized  charge  in- 
jection circuit.  An  N-channel  multiplexer  converts  N parallel  input  channels 
into  a single-channel  pulse  amplitude  modulated  (PAM)  signal.  The  input  signals 
are  synchronously  sampled  and  the  sampler  information  is  entered  into  a unique 
spatial  and  temporal  position  in  the  CCD  delay  line.  Applications  include  the 
multiplexing  of  many  sensor  input  channels  (e.g.,  electro-optical  sensors, 
acoustical  sensors)  into  a single  video  output  channel. 

Figure  4-19  illustrates  the  basic  20  input  PI/SO  CCD  building  block  with 
pin  diagram  shown  in  Figure  4-17.  Figure  4-20  shows  an  overall  photomicro- 
graph of  the  PI/SO  block.  Figure  4-21  shows  the  SI/SO  input  part  of  the  PI/SO 
block  prior  to  aluminum  metalization  with  the  polysilicon  electrodes  defined. 
Figure  4-22  shows  a plotter  schematic  and  pin-diagram  labeling  of  the  PI/SO 


(PI)  20  INPUTS  WITH  STABILIZED  CHARGE  INJECTION 


OUTPUT 
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Figure  4-19.  PI/SO  (20  Input)  CCD  Basic  Building  Block 
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Figure  4-20.  Photomicrograph  of  PI/SO  Block, 
block  and  Figure  4-23  is  a photomicrograph  of  the  PI/SO  block  after  aluminum 
metalization.  The  PI/SO  block  has  stabilized  charge  injection  at  each  of  its 
inputs  with  parallel  injection  at  alternate  stage  delays  to  minimize  inter- 
channel crosstalk  by  the  insertion  of  an  ac  reference  for  subsequent  removal 
of  input  variations  in  threshold  voltage. 

4.5.3  Time  Delay  and  Integration  (TDl)  Operation 

The  PI/SO  block  may  also  be  operated  in  the  time  delay  and  add  or  integrate 
(TDl)  mode  to  provide  unique  signal  processing  functions.  In  this  mode,  the 
various  parallel  inputs  are  combined  in  an  additive  manner  to  realize  an  improve- 
ment in  S/N  ratio  similar  to  the  distributive  floating  gate  amplifier  (DFGA). 

The  basic  idea  in  a DFGA  and  a TDl  CCD  is  to  provide  additive  amplification  of  the 
signals,  whereas  the  noise  is  increasing  in  an  rms  manner.  A DFGA  consists  of  two 
CCD  operating  in  parallel,  a TDl  CCD  and  a "signal  source"  CCD  (which  is  a serial 
IN/parallel  out  CCD)  which  provides  the  input  signal  for  the  DFGA's  TDl  CCD.  Thus, 
for  N stage 'DFGA  or  TDl  CCD  (i.e.,  N parallel  inputs)  the  output  signal  will  be  N 
times  larger  and  the  noise  (assuming  uncorrelated  noise  sources)  will  be  N times 
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—22.  Plotter  of  PU/SO  Block  with  Pin  Number  Designations 
(See  also  Figure  2-16) 


larger  with  an  overall  improvement  of  the  in  S/N.  Figure  1+-2U  illustrates 
an  example  of  the  TDI  mode  in  which  the  parallel  inputs  are  all  tied  together 
and  a 12-bit  data  window  opened.  The  serial  output  is  increasing  linearly 
as  the  analog  bits  are  added  to  one  another  until  the  input  data  window  is 
closed  or  until  the  maximum  dynamic  range  of  the  device  is  exceeded.  Figure 
b-25  illustrates  the  successive  addition  in  the  TDI  mode  for  6 parallel 
inputs  with  a 100-Hz  sinusoidal  signal  sampled  at  fc  = 100  kHz.  The  serial 
and  parallel  clocking  in  the  TDI  mode  are  synchronous  to  provide  the  additive 
signal  feature. 

U.5.U  Serial  In/Parallel  Out  (SI/PO) 

The  SI/PO  block  features  INDEPENDENT  nondestructive,  low-impedance 
voltage  readouts  of  the  analog  signals  at  specified  locations  or  taps  corres- 
ponding to  various  delays  through  the  CCD  shift  register.  In  general,  the 
signal  voltage  at  each  tap  may  be  multiplicatively  weighted  by  conductance 
to  give  a current  proportional  to  the  PRODUCT  of  the  signal  voltage  by  the 
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Figure  h-2h.  TDI  Operation  for  a Constant  Input  Signal  Common 

to  All  Parallel  Inputs 
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Figure  A— 25 • TDI  Operation  for  a Sinusoidal  (100  Hz)  Signal  Applied 
to  6 Parallel  Inputs  of  a PI/SO  Device  (f  = 100  kHz) 


weighting  conductance.  Summation  of  the  product  currents  provide  such 


functions  as  transversal  filtering,  correJ a n,  or  sampled  data  smoothing/ 


interpolation  for  line  arrays.  Tv^-iime  :'onal  weighting  mat-^'ces  driven 


by  the  independent  low-impedance  taps  of  %'  a SI/PO  block  can  give  discrete 


Fourier  transforms,  filter  banks,  or  multiple  cross  correlators.  Figure 
U-26  illustrates  a photomicrograph  of  a SI/PO  block  (N  = 20  outputs)  which 


uses  a floating  clock  electrode  sensor  at  alternate  stage  delays  along  the 


CCD  delay  line.  This  permits  the  sensing  of  a "reference-only"  and  "signal  and 


reference"  signals  whose  difference  yields  the  net  "signal"  and  eliminates  the 


nonuniformities  in  the  SI/PO  structure.  Using  such  a format,  numerous  taps  with 


multipli  -tive  analog  weighting  can  be  accommodated  on  a small  CCD  area  without 


signal  amplitude  degradation  due  to  stray  parasitic  capacitance.  Paralleling 
SI/PO  blocks,  each  with  independent  nondestructive  read  out  voltage  taps. 


f > 


Figure  4-26.  Photomicrograph,  of  SI/PO  CCD  Basic  Building  Block 

can  provide  the  analog  voltage  signals  needed  by  multipliers  to  achieve 
CCD  real-time  analog  correlation.  Use  of  programmable  conductances 
such  as  the  nonvolatile  MNOS  type  or  conventional  MOS  type  permit  such 
device  applications  as  adaptive  transversal  line  equalizer  or  programmable 
matched  filter  (or  correlation  detector)  for  secure  voice/data  communications 
systems. 


4.5.5  SI/PO  Transversal  Filters 

Transversal  filters  may  be  mechanized  with  tapped  delay  lines  as  shown 
in  Figure  4-27.  They  are  characterized  by  the  summation  of  fractions  of 
present  and  past  input  signals  and  the  absence  of  feedback.  This  kind  of 
filter  is  also  referred  to  as  a feed-forward,  longitudinal  or  nonrecursive 
filter.  The  output  of  the  filter  is  the  sum  of  signal  samples  delayed  by  a 
different  number  of  equal  increments  of  time,  and  each  of  which  may  be 
weighted  by  a different  multiplicative  constant.  A transversal  filter  having 
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a finite  number  of  taps  also  has  a finite  impulse  response  so  that  it  may  also 
be  referred  to  as  a FIR  filter.  Furthermore,  the  absence  of  feedback  paths 
prevents  (frequency  domain)  poles  in  the  transfer  function  of  the  transversal 
filter  so  that  all  frequency  response  shaping  is  done  by  judicious  location  of 
the  N (frequency  domain)  zeroes  of  a filter  having  N-l  tapped  delay  stages. 
Transversal  filters  are  more  tolerant  of  arithmetic  errors  in  the  tap  weights 
than  are  the  recursive  filters  (which  require  feedback).  The  transversal 
filter  can,  in  theory,  be  used  to  synthesize  any  linear  function.  In  general, 
the  signal  input,  tap  weights,  and  signal  output  are  complex,  considering 
the  frequency  domain  features  of  the  transversal  filter.  Low  pass  (LP)  and 
high  pass  (HP)  filters  as  well  as  bandpass  (BP)  and  band  stop  (BS)  filters 
may  be  synthesized.  The  tap  weights  for  LP  and  HP  filters  are  real,  so  that 
inphase  and  quadrative  signal  channels  can  be  independent;  whereas,  the  tap 
weights  for  image  rejection  BP  and  BS  filters  are  complex,  which  requires 
"cross-coupling"  between  the  inphase  and  quadrature  signal  channels. 


The  bandwidths  and  center  frequencies  are  normalized  to  the  unit  delay,  T, 
shown  in  Figure  4-27  so  that  a given  filter  may  be  scaled  from  one  center 
frequency  and  bandwidth  to  another  by  simply  changing  T.  It  follows  then 
that  the  center  frequency  and  bandwidth  are  limited  by  the  minimum  and 
maximum  values  of  T.  These  filters  have  a linear  phase  versus  frequency 
characteristic  provided  that  the  tap  weights  (amplitudes)  have  conjugate 
symmetry  about  the  center  of  the  weighting  array. 

4.5.6  Time  Domain  Characteristics 

Considering  the  performance  of  the  transversal  filter  in  the  time  domain, 
correlation  (or  convolution)  may  be  performed  by  considering  the  W^'s 
Figure  4-28  to  be  the  time  reference  function.  A single  impulse  applied  at 

l 

the  input  to  the  transversal  filter  will  propagate  through  the  delay  sections 
and  produce  an  output  proportional  to  the  tap  weight  of  each  stage  in  turn; 
this  describes  the  impulse  response  of  the  filter.  Matched  filters  may  be 
implemented  by  selecting  tap  weights  which  are  the  time  inverse  of  the 
known  input  signal.  Other  functions  such  as  coding,  interpolation,  centroid- 
ing,  and  single  sideband  modulation  (Hilbert  transform)  may  be  performed 
with  the  tapped  delay  line  and  summed  weights  which  represent  the  transver- 
sal filter  in  Figure  4-27. 

An  ideal  transversal  filter  would  have  several  (unattainable)  features: 

a.  Exact  tap  spacing  in  time 

b.  Loss-less  and  distortion-less  transmission  of  the  signal  in  the 
delay  line 

c.  Exact  loss-loss  tap  weights 

d.  Infinite  linear  dynamic  range  for  delay  line  and  taps 

e.  Immunity  to  environmental  effects. 

While  CCD-based  transversal  filters  are  not  perfect  in  any  of  these  re- 
spects, the  tap  spacing  can  be  made  significantly  better  than  attainable  with 
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Figure  4-28.  Uniform  Time  Function 
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any  previous  analog  technique.  The  effective  tap  spacing  is  determined  by 

. 

the  clock  which  samples  the  signal,  so  that  the  spacing  may  be  nearly  as 
accurate  as  a crystal  oscillator's  accuracy.  If  a sample  and  hold  operation 
is  done  external  to  the  CCD  shift  register,  then  the  shift  register  clocks  are 
noncritical.  If  the  sampling  operation  is  done  by  the  CCD  input  stage,  then 
the  shift  register  clocks  must  have  appropriate  stability.  CCD  shift  registers 
are  also  immune  to  reflections  which  have  previously  been  a problem  in  analog 
delay  lines. 

1*.5*7  Frequency  Domain  Characteristics 

Important  frequency  domain  characteristics  of  transversal  filters  are: 

a.  Bandwidth 

b . Passband  ripple 

c.  Skirt  sharpness  (transition  bandwidth,  or  resolution) 

d.  Sidelobe  level  (peak,  average) 

e.  Phase  versus  frequency. 

All  of  the  amplitude  versus  frequency  characteristics  may  be  traded  off  against 
one  another,  but  the  number  of  delay  elements  and  taps,  N,  impose  an  ulti- 
mate  limitation  on  the  narrowest  possible  band  of  the  response,  whether  it  be 
passband,  reject  band,  or  transition  band  (skirt). 

Since  the  frequency  response  is  the  Fourier  transform  of  the  impulse  re- 
sponse  of  a linear  network,  which  is  exactly  described  by  the  taps  of  a trans- 
versal  filter,  simple  transform  pairs  can  be  used  to  illustrate  some  of  the 
tradeoffs.  First  consider  the  uniformly  weighted  time  sequence  of  Figure  U-28(a) 
and  its  resultant  frequency  response  in  Figure  U-28(b).  Here  the  total 
duration  of  the  impulse  response  (weighting  array)  in  N delay  elements  times 
the  elemental  delay  T,  and  the  bandwidth  of  the  familiar  (sin  x/x)  frequency 
response  is  approximately  1/NT.  Other  important  characteristics  of  this 
frequency  response  are  a total  mainlobe  width  of  2/NT  and  peak  sidelobe 

f.4 

response  of  -13.6  dB,  which  are  the  narrowest  mainlobe  and  (in  a practical 
sense)  the  worst  sidelobes  attainable  witl1  1111  N point  impulse  response. 
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This  is  a reasonable  starting  point  in  considering  a tradeoff  between  sidelobe 
levels  and  bandwidth  (resolution)  since  -13.6  dB  sidelobes  are  rarely  adequate 
for  any  filter.  The  transform  pair  of  Figure  4-29  represent  the  Hamming 
window  function.  In  this  case,  much  lower  sidelobes  are  obtained  by  the  use 
of  a smoother  time  domain  function  but  at  the  price  of  twice  the  mainlobe 
width.  Many  other  weighting  functions  have  been  cataloged. 

4.5.8  Correlation  and  Convolution 

In  addition  to  filtering,  the  SI^O  block  provides  convolution  with  the  tap 
weight,  as  we  have  discussed  for  the  PI/SO  block,  and  correlation  under  the 
direction  of  the  input  signal  is  reversed  as  illustrated  in  Figure  4-30.  The 
powerful  operations  performed  with  this  structure  should  lead  to  the  replace- 
ment of  many  digital  multiples  or  adds  currently  required  to  achieve  8-10  bit 
accuracy. 

4.5.9  Nondestructive  Readout  (NDRO)  of  the  CCD  Analog  Signal 

In  the  realization  of  a transversal  filter,  we  must  provide  a low  loss 
analog  delay  line  (namely,  the  CCD)  and  NDRO  of  the  signal  for  subsequent 
weighting  and  summation.  Two  approaches  have  been  discussed  in  the  literature: 

a.  Surface  Potential  Sensing  with  a Floating  Diffusion 

b.  Displacement  Charge  Sensing  with  a Floating  Clock  Electrode  Structure. 
The  first  method  uses  a floating  diffusion  under  the  CCD  storage  electrode  to 
sense  the  surface  potential,  which-  is  given  by  the  expression, 

*s  * v0  * V1  - <v0  * 2vo  V1/2  (Ul) 
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where  Cq  is  the  oxide  capacitance /unit  area  under  the  storage  electrode,  N 
the  substrate  doping,  V the  electrode  voltage,  and  V the  flat-band  voltage. 
The  floating  diffusion  contacts  a gate  electrode  of  a MOS  amplifier  and  is 
reset  by  a MOS  switch.  A major  problem  with  surface  potential  sensing  is 


Figure  h-30.  SI/PO  Functions 

the  modulation  of  the  storage  well  barrier  height  by  adjacent  fringe  fields.  The 
modulation  of  these  barrier  heights  is  responsible  for  the  poor  transfer  efficiency 
in  conductively  coupled  CCD  and  Bucket  Brigade r Cfevices  structures.  Recent  aware- 
ness of  such  problems  has  resulted  in  the  introduction  of  adjacent  "protective 
electrodes"  which  are  held  at  a fixed  potential  to  counter  the  varying  fringe 
fields  under  s-orage  electrode.  The  voltage  on  the  gate  of  the  MOS  amplifier  is 


related  to  the  signal  charge  by  the  expression. 
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where  C^  is  the  external  capacitance  attached  to  the  sensing  node  and  C0  is 
the  semiconductor  space  charge  capacitance.  The  same  expression  is  valid 
for  the  floating  clock  electrode  sensor  (FCLS)  shown  in  Figure  ^-31.  The 


FCLS  sensor  does  not  require  a diffusion  beneath  the  storage  electrode  and 


Figure  4-31.  Floating  Clock.  Electrode  Senaor  (FCES)  Circuit  for 
Nondestructive  Signal  Charge  Sensing 

is  therefore  not  subject  to  fringe  field  nonlinearity  effects.  Because  of 
relative  immunity  of  the  FCLS  to  fringe  field  effects  and  barrier  height  modulation 
we  believe  the  best  method  to  sense  change  in  a CCD  is  the  FCLS.  The  semiconductor 
space  charge  capacitance  is  related  to  the  surface  potential  g,  and  hence  signal 
charge,  by  the  expression 
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Combining  equations  (4-1),  (4-2),  and  (4-3)  provides  a relationship  between 
signal  charge  and  voltage  on  the  gate  of  the  MOS  amplifier . 
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A new  concept  has  been  developed  in  the  analysis  of  the  PI/SO  circuit. 

The  time  delay  and  integration  (TDI)  mode  of  operation  performs  progressive 
or  successive  additions  of  the  signal  as  shown  in  Figure  4-32.  If  the  inputs 
are  preweighted  as  indicated  in  the  figure,  then  the  output  of  the  serial  delay 
line  is  simply  given  as 
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assuming  we  inject  at  each  stage  delay.  Actually,  we  inject  at  alternate  sites 
to  permit  the  incorporation  of  a reference  signal  (ac  zero)  together  with  the 
actual  sampled  signal.  The  above  expression  is  identical  to  the  transversal 
filter  response  of  a SI/PO  block  as  shown  schematically  in  Figure  4-32.  Thus, 
the  PI/SO  block  can  perform  the  same  functions  as  the  SI/PO  without  the 
need  for  nondestructive  tapping,  amplification,  and  buffering  at  each  tap 
location.  There  are  speed  advantages  since  the  electrodes  can  be  placed 


SI/PO  Simultaneous  Summation 
(Transversal  Filter) 


PI/SO  Progressive  Summation 
(Time  Delay  and  Integration) 


1 


)•**  ; 

I: 

l-'.j 

I’JeJ 

5 j 


; i 

■ 

A 


Figure  4-32.  Comparison  of  PI/SO  and  SI/PO  Filter  Methods 
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close  together;  however,  the  biggest  advantage  appears  to  lie  in  the  control 
of  offsets  with  no  amplifier  gain  variation  to  compensate.  The  disadvantage 
is  the  need  to  provide  a tapered  width  to  handle  the  dynamic  range;  however, 
this  is  not  a big  problem  since  the  chip  size  remains  almost  unchanged 
(i.e.,  clocks  and  bonding  pads  take  up  more  room  than  channel  widening). 


In  addition  to  the  filter  operation  the  mathematical  operation  given  by 
equation  (4-5)  is  described  by  a convolution  of  the  tap  weights  with  the 
delayed  input  serial  e^R  (t-kT).  If  the  tap  weights  are  controlled  electrically, 
then  the  structure  shown  in  Figure  4— 32(a)  becomes  a reprogrammable  filter/ 
correlator . 
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